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Abstract 

Errors in the localization of nearby virtual objects pre- 
sented via see-through, helmet mounted displays are ex- 
amined as a function of viewing conditions and scene 
content Monocular, biocular or stereoscopic presentation 
of the virtual objects, accommodation (required focus), 
subjects' age, and the position of physical surfaces arc ex- 
amined. Nearby physical surfaces are found to introduce 
localization errors that differ depending upon the other ex- 
perimental factors. The apparent physical size and trans- 
parency of the virtual objects and physical surfaces re- 
spectively are also influenced by their relative position 
when superimposed. Design implications are discussed. 

1. Introduction 

The following four experiments explore such phe- 
nomenon by examining subjects' ability to adjust the dis- 
tance of a physical pointer to match that of a nearby vir- 
tual object. The object is generated by a high perfor- 
mance, computer graphics system and presented by a 
head-mounted, see-through display. This localization task 
was selected since it is close to the visual-manual manip- 
ulation expected of users of virtual objects in numerous 
possible applications from surgery to mechanical assem- 
bly on a production line . In these applications virtual 
objects are proposed to become 3D guides that instruct 
and aid manual interaction with the world by dynamically 
helping the user position his hand. Thus, these virtual 
objects must be accurately and reliably localized by the 
user. For this reason, and because its precision and accu- 
racy can be easily and reliably measured, the following 
experiments study the localization of virtual objects with 
a real pointer. Preliminary testing showed that our sub- 
jects could set a mechanically displaced, physical pointer 
to match the distance of physical targets with several mil- 
limeter accuracy and that this accuracy corresponded to 
their ability to match target distances with their fingers. 
Most previous work has examined system configuration 
to optimize display performance (e.g. [11 [2] ) but only 
Rolland’s [3] laboratory has systematically investigated 
user localization performance. 

The initial experiment below examines the effects of 
three different viewing conditions on the subjects' accu- 
racy of placement of the physical pointer under a virtual 
object. The conditions are: 1) monocular, 2) biocular or 
3) stereoscopic viewing. These represent a range of cost 
and image fidelity, i.e. the completeness with which an 
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objects physical characteristics are presented. The 
monocular condition presents a virtual image of an object 
much as monocular helmet mounted sights do and repre- 
sents a minimal hardware/software rendering cost for such 
virtual image displays. But the monocular virtual images 
it presents are subject to visual suppression due to binoc- 
ular rivalry. The biocular condition, which presents two 
identical virtual images to the subjects' left and right 
eyes, can avoid the rivalry problem. The images it pre- 
sents are projected from a cyclopean position between the 
viewers eyes but are shifted to allow unstressed fusion at 
a selected distance (See Figure 1). This condition halves 
the rendering cost with respect to stereo displays, but 
doubles the head-mounted display hardware requirements 
with respect to monocular displays. It presents a pattern 
of disparity approximating that of a transparent fiat dis- 
play surface and represents an intermediate cost system 
with potentially more stable image brightness. The third 
condition, conventional stereo display with parallel view- 
ing vectors, presents the highest spatial fidelity but dou- 
bles both the rendering and hardware display cost com- 
pared to the monocular condition. 

Because the monocular and biocular viewing conditions 
degrade the fidelity with which distance is presented, we 
expect that the stereoscopic display should support the 
most accurate localization. The biocular display may, 
however, provide a competitive alternative for virtual ob- 
jects with relatively little internal depth by avoiding the 
potential problem of binocular rivalry. The first experi- 
ment provides a descriptive study of relative localization 
accuracy of virtual objects presented via the alternative 
viewing conditions. Since observer age and accommoda- 
tive demand (required focus) could be expected to interact 
and influence localization accuracy, these display charac- 
teristics are also examined in the first study to develop de- 
signer guidelines for the adjustment of focus and selection 
of personnel to use virtual object displays . Subsequent 
studies consider the effect of the introduction of a nearby 
physical surfaces on the localization of the virtual object, 
identify a phenomenon that introduces errors into such lo- 
calization, and explore two alternative explanations for 
the phenomenon. 

2. Experiment 1 
2.1 Methods 

All experiments reported in this paper used a custom, 
head-mounted, see-through display, called an electronic 


haploscope by the authors, capable of presenting a 20-30° 
diameter circular monocular field to each eye with variable 
monocular overlap. In the following experiments the 
system was used at 100% overlap and 20° field of view. 
The display system used two vertically mounted Citizen 
1.5 * 1000 line miniature Cathode Ray Tube’s (CRT) in 
National Television Standards Committee (NTSC) mode 
which were driven by a Silicon Graphics (SGI) computer 
(4D/210GTXB) through custom video conditioning cir- 
cuits. For the simple 3D imagery used in the following 
experiments, the computer could maintain a 15 Hz graph- 
ics update rate. The CRT images were infinity collimated 
by standard glass telescope eyepieces (Erfl 32 mm and 
Ploessl 42 mm) mounted directly under the CRTs. After 
the signal transformation from the RGB to NTSC, indi- 
vidual pixels which corresponded in the current configu- 
ration to 3 aremin horizontal resolution measured from 
subjects' eyes were easily discriminated. The collimated 
light could be modified by lenses and rotating prisms 
from a standard optometric trial lens set that allowed pre- 
cise positioning with at least 5 aremin resolution of the 
separate left and right images and allowed variation of the 
accommodative demand for each eye. The images were re- 
layed to the subjects eyes by custom, partially silvered 
(15%) polycarbonate mirrors mounted at 45° directly in 
front of each eye. The left and right viewing channels 
could be mechanically adjusted between 55 mm and 71 
mm separations for different subject's interpupilary dis- 
tances. The video signal conditioning also allowed lateral 
adjustment of the video frame. Consequently, the display 
system can precisely position the center of each graphics 
viewport directly in front of the eyes of all subjects for 
bore-sight alignment. 

The entire display system, built around a snug fitting, 
rigid headband, is intended to be worn by a freely moving 
subject and weights between 0.77 and 1.1 kg. depending 
upon configuration. In the lightest configuration the mo- 
ments of inertia have been measured when mounted on an 
erect head to be: 0.0782 kg-m 2 vertical axis, 0.0644 
kg-m 2 longitudinal axis, 0.0391 kg-m 2 lateral axis. In 
all of the experiments described below, the band was fit- 
ted to each subjects head and then supported by a special 
pivoted mount at the end of a 1.8 m table. This mount 
restricted horizontal movement but allowed some pitch 
movement. Subjects sat at this end during the course of 
each experiment. The mount and chair were adjusted so 
that the virtual objects could be presented at eye level. 
Lateral head movement was restricted during all the fol- 
lowing experiments but a residual pitch of ±10° was al- 
lowed for subject comfort. In practice the subjects were 
monitored by the experimenters to keep their heads ap- 
proximately level at an individually selected orientation 
during the course of the experiments. 

A monocular, biocular or stereoscopic virtual image of 
an upside down, axially rotating (-3 rpm) pyramid was 
presented at a distance of 58 cm away from the subjects 
eyes by a head-mounted see-through display. It was seen 


against a grey, cloth covered wall 2.2 m from the sub- 
jects. Preliminary experiments examining varying the ro- 
tation rate of the pyramid for each trial showed that such 
variation had no effect on the localization of the virtual 
image. The reference distance of 58 cm was chosen for 
the experiment because it corresponded to a possible 
working distance for several industrial applications of in- 
terest to the authors. All displays were operated under 
moderate indoor artificial illumination (approximately 50 
lux). The virtual image was presented with either 2 
diopters accommodative relief or at optical infinity. The 
stereo display was, however, calibrated (see below) over a 
range of 30 to 1 10 cm. The monocular display was sim- 
ply the stereo channel that corresponded to the subjects’ 
dominant eye. The biocular display was produced by po- 
sitioning the graphics eye point midway between the sub- 
jects eyes. The left and right images were identical copies 
of this view but were shifted laterally so that when the 
subjects eyes converged to the centers of each view-port 
they would have 0 disparity relative to the reference con- 
vergence point 58 cm. The plane of 0 disparity was thus 
set so that the subjects could easily fuse the images when 
converged at 58 cm. This technique was used in general 
for all biocular stimuli at different depths which were ex- 
perimentally interjected as described below. Though no 
keystone correction was applied for the distortion caused 
by the image shift, the disparity pattern produced in this 
biocular image closely approximates that of a flat image 
of the target as if it were drawn on a transparent projec- 
tion surface at the simulated convergence distance. 
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The wire-frame pyramid had a nominal 10 cm base and 
5 cm height. The width of the wire frame lines were 
about 9 aremin. The depicted size of the presented virtual 
object was randomly scaled from 70 to 130% of its nom- 
inal size for each trial to interfere with subjects' possible 
use of angular size as a depth cue. The lines of the wire 
frame an all other computer generated lines had a lumi- 
nance of about 65 cd/m 2 and were seen against the gray 
cloth background of 2.9 cd/m 2 with visible vertical 
seams. While the presented luminance did not approach 
that used in aircraft heads up displays which must be vis- 
ible against a 30,000 cd/m 2 background, it was just ade- 
quate for indoor work against most colored surfaces and is 
at least 3 times brighter than the luminance available in 
off-the-shelf, see-through head-mounted displays such as 
the IGlasses™ formerly made by VIO. 



Four out of every 30 judgments were based on unana- 
lyzed random variations in the depicted depth of the pyra- 
mid. This variation was introduced to help insure that the 
subjects did not notice that the same depicted depth was 
repeated. However, the major factor masking the repeti- 
tion of the depicted depth were the perceptual effects caus- 
ing changes in the apparent depth with different viewing 
conditions. Since the viewing conditions could be unob- 
trusively intermixed and no feedback was given to the 
subjects, there was no way for them to tell that the de- 
picted distance was not in fact changing. In fact, no naive 
subject in any of the following experiments reported 
noticing the repetition of the target depth. 

The subjects’ task was to use a method of adjustment 
to position the binocularly visible, physical cursor, a 
yellow-green light emitting diode (LED) (about 20 cd/m 
) pointer, shaped like a pyramid (base 0.5 cm, height 1 
cm) into vertical alignment with the apex of the inverted 
pyramidal virtual object. The physical cursor was moved 
on a rail by a chain and gear system and positioned about 
2 degrees of visual angle below the virtual object. The 
distance to the pointer was automatically recorded through 
use of a shaft-encoder interfaced to the display computer. 
The adjustment was self paced, but subjects were encour- 
aged to take between 15 and 30 seconds for each adjust- 
ment and were allowed to take breaks at one half to 1 
hours intervals as needed. As part of the standard proce- 
dure for use of human subjects, all subjects were informed 
they could terminate the experiment at anytime if they 
experienced any undo discomfort and were asked at the end 
if they experienced any viewing difficulties seeing the vir- 
tual objects. 

The haploscope display system was adjusted by 
monocular superimposition of reference virtual images on 
an 18 cm diameter circumscribed circle presented at a dis- 
tance of 2.2 m. In addition to position adjustment this 
allowed adjustment of the field of view angle of the 
graphics system to match the total magnification of the 
system. This was done separately for each eye and for 
each subject before the experiment was started. Thus, we 
could account for any changes due to variation of ac- 
commodative demand and corrective lenses that might be 
worn by the subjects. The subjects' interpupilary dis- 
tances (IPD) were measured with a binocular-type viewing 
device with digital readout (Varilux Model: Digital 
CRP). All displays and algorithms were adjusted to re- 
flect the measured IPD values. 

Preliminary test results for virtual targets placed be- 
tween 33 and 108 cm showed that within the full range of 
adjustment used for the experiment subjects using a 
stereo display could align the cursor within ±0.3 cm of 
the depicted virtual object target depth [4] . The distance 
responses were completely linear, unbiased and unskewed 
and were conducted in the same full room illumination as 
the experiment. Similar tests of the biocular viewing 
condition showed equally linear responses. Tests in the 


monocular condition showed, expectedly, inconsistent be- 
havior. In further examination of the localization tech- 
nique pilot subjects were asked to use the pointer to 
match (he depth of physical targets. These tests showed 
linear, unbiased, unskewed estimates with statistical 
ranges of ±0.15 cm about depicted physical distances for 
targets used in the experiment. 

Ten subjects, five young (15 - 29 yrs.) and five older 
(38 - 47 yrs.) participated in the experiment. Subjects in 
the older group could be presumed by population data to 
be at least early presbyopes [5]. All but one young and 
one older subject (i.e. the authors) were naive with re- 
spect to the purpose of the experiment. The others were 
either paid subjects recruited through the Ames Bionetics 
contractor or were laboratory personnel. All subjects 
were screened on the Bauch & Lomb Orthorater stereo 
tests for stereoacuity better than 1 aremin. Subjects who 
normally wore prescription spectacles were allowed to 
wear them during the screening test and during the exper- 
iment. During pilot testing for the experiment, inadver- 
tent errors of 0.1 - 0.2 cm in modeling of subjects inter- 
pupilary distances in the graphics simulation produced 
easily noticeable artifacts. Precise stereo or biocular pre- 
sentation of virtual objects evidently requires measure- 
ment and modeling of interpupilary distance with an accu- 
racy on the order of ±0. 1 cm. 

Viewing Conditions (monocular, biocular, and stereo- 
scopic) were crossed with Accommodation ( 0 or 2 
Diopters) and nested within Age groups. The experiment 
used a blocked design in which blocks of 5 replications 
of a given condition were presented for each of the three 
viewing conditions producing uninterrupted 15 judgment 
sequences. The sequence of viewing conditions were ran- 
domly assigned to each subject and thereafter systemati- 
cally permuted after each set of 3 viewing conditions were 
presented. In general, it was possible to switch the view- 
ing conditions solely through software. Thus, the sub- 
jects were generally unaware which viewing condition 
was presented and the perceptual variation in apparent 
depth caused by variation in view condition was readily 
interpreted by them as variation in depicted depth. The 
viewing conditions were blocked for a given accommoda- 
tive demand which was switched by interrupting every 15 
trials to change viewing lenses. The order of presentation 
of accommodative demand was permuted within subjects 
and balanced across subjects. 

2.2 Results 

Analysis of variance (ANOVA) showed that the view- 
ing conditions had a major effect on the bias of the sub- 
jects distance judgments (F= 15.580, df =2, 16, p < 
.001). The mean stereoscopic and biocular localizations 
were almost completely correct, but a judgment bias ap- 
peared as an overestimate when the stereo depth cues as- 
sociated with the virtual object were removed by monocu- 
lar viewing. This effect interacted with accommodative 



demand and age, as indicated in Figure 2 (F-7.76, df - 2* 
16, p < .004). All other effects are related to this three 
way interaction and will not be discussed individually. 
No subjects reported any difficulties seeing the virtual ob- 
jects during the experiment. 
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Figure 2: Effect of Age/Accommodative on localization. 

2.3 Discussion 

The results of the ANOVA plotted in Figure 2 show 
that when depth cues to the virtual object are degraded to 
monocular conditions, judgments of its distance drop back 
towards the distance of the background wall at 2.2 m from 
the subjects. A phenomenon that could explain the in- 
creased judged target distance in the monocular condition 
is the specific distance effect which causes visual targets 
of unknown physical size presented with weak or ambigu- 
ous depth cues to tend to appear in visually impoverished 
environments about 2-3 meters away [6]. This distance 
effect is also associated with tonic accommodation and 
vergence which relax to approximately 1-2 meters in the 
absence of distance cues [7]. Changes of convergence to a 
more distant resting position could cause the localization 
of the virtual object to recede from the viewer. But since 
these effects are generally seen when targets appear 
against featureless backgrounds they are not likely impor- 
tant for the present results. 

Another phenomenon probably more relevant is the di- 
rect effect of change in ocular convergence on perceived 
depth [8] [9]. It is the kind of result to be expected if the 
subjects' actual convergence was driven or attracted by the 
wall which provided a visually sharp, textured cloth with 
vertical seams at 2.2 m. In the monocular condition, the 
only source of information that the virtual object is lo- 
cated any particular distance in front of the background is 
provided by accommodative demand. It is therefore not 
surprising that when 2 diopters of accommodative demand 
was provided, only the subjects young enough to respond 
to this cue were able to localize the virtual object approx- 
imately correctly. In these subjects the accommodative- 
convergence reflex allowed them to maintain convergence 
on the virtual object. Older subjects unable to respond to 
accommodative cues only have the disparity information 


provided by the background cloth to control their ver- 
gence. Thus, they would still diverge to fuse the back- 
ground and the monocularly presented virtual objects 
would still appear toward or on the background wall. 

That the monocular virtual objects are not judged to be 
exactly on the wall, reflects the response bias within the 
experiment originating from constraints on movement of 
the physical cursor, unavoidable guesses the observers 
may have made about the approximate size of the object, 
and the possibility that the observer converged to other 
closer objects such as the pointer and its supports which, 
though darkened to be less conspicuous, were still visible 
against the background. In fact, if the observers were to 
hold their eyes completely still, the monocular viewing 
situation is very similar to that of viewing a monocular 
afterimage in a demonstration of Emmerts Law, the clas- 
sic observation that an afterimage often appears at the dis- 
tance of a physical surface against which it is projected 
[10]. The readily available correct disparity information 
in the biocular and stereo conditions, however, provide 
the missing cue that allowed all observers to correctly 
judge the distance to the virtual object. 

Finally, since no difficulties seeing the virtual objects 
were reported during the experiment, we find no evidence 
that binocular rivalry interfered with subjects' ability to 
see the virtual objects in the monocular condition. The 
finding is consistent with all observations we have made 
of the monocular wire-frame virtual objects during prepa- 
ration for the experiment. Apparently, the high contrast 
& motion of the objects we have examined easily over- 
comes any binocular rivalry that might be present. 

3. Experiment 2 

Experiment 1 examined the effect of different viewing 
conditions on the localization of virtual objects superim- 
posed on a physical surface 2.2 m distant. But since new 
uses of virtual objects are likely to bring them closer to 
physical surfaces. Experiment 2 examines the effect of in- 
troduction of a much closer physical surface. In view of 
the interacting roles of accommodative and convergence 
in the discussion of Experiment 1, one could reasonably 
expect the introduction of a nearby real surface to cause 
the observers to localize monocularly viewed virtual ob- 
jects at the same distance of introduced surface. If the ac- 
commodative demand for the virtual object is already 
matched to its displayed distance, one would expect that 
this improvement in the accuracy of localization would be 
larger for older observers than younger ones who already 
would have accommodative cues to the virtual object dis- 
tance [11]. Accordingly, subjects of different age groups 
were used while correct accommodative demand to the vir- 
tual object was provided. In order to study the effects of 
introduction of a physical surface, subjects must first 
judge the distance of the virtual image by itself. This 
first judgment is identical to those made with the viewing 
conditions in Experiment 1 with 2 D accommodative de- 


mand and provides a chance to replicate that part of the 
experiment. 


3.1 Methods 

The virtual image stimuli used in Experiment 2 were 
identical to those in Experiment 1 for the 2 diopter ac- 
commodation condition but a new physical stimulus was 
introduced. This physical surface was a slowly, irregu- 
larly rotating checkerboard (—2 rpm) made of xeroxed pa- 
per glued on foam-core and was mechanically introduced 
along the line of sight to the pyramid as illustrated in 
Figure 3. Motion of the checkerboard was introduced be- 


cause preliminary testing showed that changes in localiza- 
tion that it produced were enhanced by motion. The 
checkerboard was a disk 29 cm in diameter with 5 cm 
black and white checks having either 1.3 cd/m^ or 17.8 
cd/m^ luminance. It was positioned so that the virtual 
image of the pyramid could be seen against the lower rim 
of the disk in order to allow the subjects to adjust the 
physical cursor to the apparent distance of the virtual im- 
age in the presence of the disk. Care was taken to be sure 
the physical cursor was below the bottom of the virtual 
object and the edge of the disk as in Experiment 1. As 
before, subjects viewed the virtual objects with either 
monocular, biocular or stereoscopic view conditions. 
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Figure 3: Experimental procedure illustration for Experiments 2, 3, and 4. Top: alignment, magnification, and in teipupU- 
^ a * ctmpnt vriHdip* initial localization of virtual object depth. Bottom: Testing conditions representing, the on or 
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3 and 4 represent use of nonius lines to detect relative convergence in Experiment 3. 


Thirteen subjects, seven young (15-29 yrs.) and six 
older (38 - 47 yrs.) participated in the experiment All 
but one young and one older subject (the authors) were 
naive with respect to the purpose of the experiment The 
others were either recruited through the Ames Bionetics 
contractor or were laboratory personnel. All subjects 
were screened for stereo vision as in Experiment 1. 

The first part of the subjects' task was to mechanically 
place the yellow-green LED pointer under the nadir of the 
slowly rotating, wire-frame virtual pyramid, which varied 
randomly in size for each trial as in Experiment 1. The 
second part of the task involved an adjustment of the 
pointer to match the pyramid's distance after the slowly, 
irregularly rotating, opaque checkerboard was introduced 
along the line of sight to the pyramid. The checkerboard 
was introduced at the previously judged distance of the 
apex of the virtual pyramid. This fact was unknown to 
all the naive subjects and remained unnoticed throughout 


the experiment. Though the virtual pyramid was also 
presented a second time at the same distance as the first 
localization, the experimental variations generally con- 
cealed this fact from the naive subjects who were led to 
believe each trial, with or without the checkerboard, in- 
volved a potentially different depicted depth. As in 
Experiment 1, the occasional introduction of unanalyzed 
sham targets at different depths enforced the naive sub- 
jects’ belief that the virtual image possibly could be dis- 
placed variously in depth for every localization. 

3.2 Results 

Analysis of the subjects' first localization the virtual 
object under the three viewing conditions with 2 diopters 
of accommodative demand closely replicate the findings 
of Experiment 1. The basic result is a significant 2- way 
interaction of view condition and age (F(2,42)=19.160 p 
< 0.0009) in which age variation effects the judged dis- 




lance only for the monocular viewing condition, with the 
younger subjects (Figure 4). 
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Figure 4: Interaction of Age and Viewing Condition 

Analysis of the offset of the mean judged distance to 
the virtual object associated with the introduction of the 
physical surface also showed a main effect of viewing 
condition (F(2,26) = 91.340, p <.0001) and a significant 
interaction between viewing condition and age (F(2,26) = 
21 . 921 , p <.0001) (Figure 4). These effects modulated 
the overall significant offset (F(l,13) = 90.623 p 
<.0001) of the judged distance to the virtual object to- 
wards the viewer which was caused by introduction of the 
physical surface. This effect appears for all viewing con- 
ditions as a closer localization of the target after interposi- 
tion of the physical surface. 

3.3 Discussion 

The first virtual object localization shown in Figure 4 
does replicate the 2 D viewing conditions in Experiment 
1 showing that the older subjects are unable to use the ac- 
commodative information to estimate the virtual object 
distance and consequently localize the virtual object erro- 
neously towards the background wall. Interestingly, in- 
terposition of the checkerboard at the judged distance of 
the monocularly viewed virtual object causes a substantial 
forward movement of the judged virtual object position 
(Figure 4). This change is what would be expected if in- 
sertion of the disparity and other cues to nearness of the 
checkerboard were to cause relative convergence with re- 
spect to the position of eyes before its introduction. 

It is important to note that the forward movement of 
the localization is from the initial judged position for 
each appearance of the isolated virtual object. Since the 
older subjects tended to initially judge the virtual object 
to be too far away, the tendency for the checkerboard to 
bring the judge distance closer was corrective 


(See Fig. 4). On the other hand, younger subjects view- 
ing monocularly did not significantly misjudge the virtual 
object distance, as evident from the error bars. For them 
introduction of the checkerboard was detrimental, causing 
the virtual object to be judged too near. 

In fact, the checkerboard insertion caused the virtual 
object to generally appear too near for the other two view- 
ing conditions which otherwise supported correct localiza- 
tion of the isolated virtual object. This was generally 
true for both age groups and suggests that introduction of 
the physical surface, the checkerboard, could be causing a 
small relative convergence under these conditions as well. 
Though disparity information for correct convergence to 
the virtual object is available under these two conditions, 
the virtual object providing this information is not of 
high visual quality. Its presentation corresponds roughly 
to 20/100 visual acuity. Under such conditions, conver- 
gence based on stereoscopic cues might not be precise and 
could exhibit a fixation disparity. If this error were an 
exofixation disparity, i.e. it would tend to the distant 
side, introduction of the high visual fidelity checkerboard 
could correct it, causing a relative convergence and associ- 
ated decrease of the judged distance to the virtual object. 
If this small corrective convergence were incompletely 
compensated due to the breakdown of distance constancy, 
errors of localization could be expected. Thus, the change 
in localization after introduction of the checkerboard could 
be due to a change in static convergence. 

4. Experiment 3 

Experiment 3 explicitly tests for such a change associ- 
ated with the change in judged distance. Attention is fo- 
cused on the stereoscopic viewing condition to see if the 
closer judged distance of the virtual object associated with 
introduction of the physical surface could be associated 
with an increase in static convergence. Since the amount 
of expected change is small, for example a change of 3 
cm from 58 cm to 55 cm, a sensitive measure of conver- 
gence is needed. Angular changes of monocular position 
of only about 3 aremin would be expected if the change in 
localization were explainable by vergence change alone. 
Such a measurement is difficult to make without encum- 
bering eye tracking technology that preserves a clear vi- 
sual field. But it is conveniently just at the display reso- 
lution of the display configuration used. Therefore, a 
technique using nonius lines on the display itself was 
adopted [12], A nonius line is a line which is broken 
into two line segments each of which is visible by only 
one eye. Such lines have typically been used to measure 
equivalent oculocentric directions to determine the posi- 
tion of the stereoscopic horopter. When the two seg- 
ments are moved laterally during a period of constant 
convergence so that they appear to be collinear, their po- 
sitions may be used to record a specific convergence posi- 
tion. 

4.1 Methods 




Since pilot studies had suggested that longer distances 
enhance the effect of introduction of the checkerboard, the 
virtual pyramid was presented at 108 cm rather than 58 
cm away from the subjects' eyes. Such an enhancement 
was deemed helpful to increase the delectability of any 
convergence. This display, otherwise similar to that used 
in Exp. 2, was operated under normal room illumination 
with one diopter accommodative relief for the virtual im- 
age. Flanking nonius lines (lower right panels of Fig. 3) 
of the same luminance and line width as the pyramid de- 
scribed in the Task section below were also occasionally 
presented to detect changes in static convergence. 

Five men and one woman with measured stereo resolu- 
tion of better than 1 aremin participated in the experi- 
ment. Some subjects had vision corrected by contact 
lenses or glasses and were able to wear their corrections 
during the experiment. Subjects’ ages ranged from 17 - 
47 and included laboratory personnel as well as paid sub- 
jects recruited by a contractor at Ames. Because of the 
computer control of the experiment it was possible to 
conduct this experiment double blind. 

The subjects task had three basic parts: 1) Localization 
of an isolated virtual pyramid and measurement of associ- 
ated static ocular convergence 2) Relocalization of the vir- 
tual pyramid in the presence of a real surface either at or 
in front of the pyramid's apparent distance. 3) 
Measurement of changes in static convergence associated 
with the relocalization. The first part of the subject's task 
was to mechanically place the LED pointer under the 
nadir of the slowly rotating, wire-frame pyramid. After 
aligning the pointer, the subjects were presented with 
two sets of vertical nonius lines just flanking the pyramid 
(Figure 3, right panel #3). These lines were then adjusted 
to appear vertically collinear, i.e. to have equal visual di- 
rections on each side of the pyramid. This adjustment 
was made by moving the lower left and right segments 
with a joystick control (See Figure 3) and effectively 
recorded the subjects' static convergence during this part 
of the experiment. Subsequent brief presentations of the 
nonius line will accordingly show how static convergence 
may have changed by revealing vertical misalignments. 
The second part of the task involved another adjustment 
of the pointer to the pyramid's depth after the slowly, ir- 
regularly rotating checkerboard was introduced along the 
line of sight to the pyramid. The pyramid was then pre- 
sented a second time at the same depicted depth in this 
new configuration. As before, the experimental varia- 
tions generally concealed this fact from the subjects so 
that they believed each trial, with or without the checker- 
board, involved a potentially different depicted depth. And 
as before, unanalyzed trials with random variation in dis- 
tance were introduced to maintain the subjects' uncer- 
tainty. 

After the second judgment of the pyramid’s depth, the 
nonius lines were flashed briefly (ca 250 msec) next to 
the pyramid while the subjects fixated it (Figure 3, right 


panel #4). Then the subjects made a forced choice indicat- 
ing whether the upper or lower pair of the flashed nonius 
lines appeared closer laterally. The eye assignments of 
each segment of the nonius lines were randomly selected 
so that the meaning of the alternative possibilities in 
terms of convergence or divergence varied randomly across 
the trials. The assignment of the lower part of the left 
nonius line and the upper part of the right line to one eye 
and the other upper-lower pair to the other eye, produced a 
differential effect doubling the relative misalignment for 
any given vergence change and increasing the sensitivity 
of the technique for detecting changes in convergence. 
The subject reported by a button press which of the paired 
nonius lines were closer. 

In fact, three different experimental conditions were 
used in the second pan of the experiment because of the 
need for a control case. In the "on" condition the 
checkerboard was mechanically introduced at the judged 
depth of the virtual pyramid object so that the pyramid 
appeared "on” the checkerboard. For the "in front" condi- 
tion the checkerboard was introduced 30 cm in front of the 
judged depth. In the control condition the second judg- 
ment was a replication of the first judgment in that the 
subject made a second judgment of the depth of the virtual 
object. But this time the subject made the forced-choice 
judgment of the nonius lines alignment without the addi- 
tion of the checkerboard. Thus, the control was identical 
to the experimental conditions except the checkerboard 
was not introduced into the line of sight. Therefore, this 
control provides an individual baseline for subjects' judg- 
ment biases and changes of their convergence during the 
course of a trial. Each condition was repeated 15 times 
for each subject in a randomized block design in which 
blocks of 5 replications of each condition were repeated. 
The 6 possible orders of the 3 conditions were distributed 
randomly across the 6 subjects in the experiment. 

The change in judged distance of the virtual object was 
analyzed in a single factor repeated measures ANOV A. 
Chi-square analyzes were conducted on each individual 
subject's distribution of judgments of conver- 
gence/divergence for each of the 3 experimental condi- 
tions. Taking the control condition as a baseline, the rel- 
ative strength of convergence could be measured by a ratio 
of the probability of convergence in each experimental 
condition to the probability of convergence in each sub- 
ject's individual control. This ratio allows control for the 
possibility that subjects might have an individual bias to 
converge or diverge simply because of a repeated presenta- 
tion of the virtual object. 

4.2 Results 

Single factor repeated measures analysis of the effect 
of superposition of the checkerboard and virtual images 
replicated the previous observations that the virtual object 
was moved closer to the viewer (F(2,10)= 7.549 p < 
0.01). Individual data are shown in Figure 5. This effect 



was somewhat stronger for the "on" condition than for 
the "in front" case and varied in strength across the 6 sub- 
jects. One subject interestingly showed no major effect. 
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Figure 5: Relative convergence vs. judged distance 


The cause of this individual subject's result is illumi- 
nated by considering all subjects' tendency to relatively 
converge during judgment of the depth of the virtual ob- 
ject in the presence of the checkerboard. This tendency is 
summarized for the experiment in Table 1 which displays 
the frequency of convergence or divergence indicated by 
the nonius judgments for all subjects in the three experi- 
mental conditions (Chi-square = 20.37, df=2, p< .001). 
The control case shows the expected 50:50 break, collaps- 
ing across all subjects, while the other two conditions 
show clear convergence, the "on" condition being some- 
what stronger. 


Convergence 

Divergence 

Total 

Judgments 

Ratio of 
Convergence 
to Divergence 

TK S3 

21 

105 

1 «5tJ 

In Front 70 

35 

105 

132 

Control 53 

52 

105 

1.00 


Tablel. Frequency of convergence to divergence 

For further analysis each subject’s individual tendency 
to converge was computed separately as the ratio of their 
probability of convergence in an exp. condition to their 
probability of convergence in the control. These ratios 
are plotted in Fig. 6 for each subject. Since the control 
was used as reference, all ratios for the control condition 
are 1. A 2x2 Chi square contingency was also computed 
to compare the distribution of convergence and divergence 
for each exp. condition to that of the control. This was 
done separately for each subject for whom statistically 
significant differences are indicated by asterisks (Fig. 5). 


4.3 Discussion 

The individual subject's localization errors in Figure 5 
are sorted by the size of the change in the judged position 
of the virtual object for the "on" condition. These results 
can then be compared with the ratio of the convergence 
probabilities. As is clear from the figure, the two mea- 
surements are almost perfectly correlated across the sub- 
jects. The only subject not to show a displacement of the 
virtual object caused by the checkerboard, also is the only 
one to show essentially no relative convergence. The 
subject showing the largest displacement due to introduc- 
tion of the checkerboard is also the one with the strongest 
tendency to converge. The results for the "in front con- 
dition show a weaker apparent displacement of the virtual 
image but also show a correlation of convergence tenden- 
cies and changes in localization. The correlation of rela- 
tive convergence with magnitude of displacement for the 
’’on" and "in front" conditions across subjects and condi- 
tions is; in fact, r= 0.894 (df=10; t=6.31, p < .002). 
These results generally support the supposition that the 
change in judged depth could be related to a change in 
convergence, but the mechanism underlying this change 
remains to be clarified. Correction of a fixation disparity, 
for example, by introduction of a high resolution physical 
stimulus, for example, is not the only possible mecha- 
nism. 

One other possibility is that change in convergence is 
due to so-called perspective [13] or proximal vergence 
[14]. These phenomena are changes in convergence due 
to changes in the apparent nearness of objects. They pro- 
vide evidence that spatial interpretations of the distance 
of a visual image themselves can simulate the vergence 
system. Accordingly, the results from the present exper- 
iment, while showing that there is a clear oculomotor re- 
sponse associated with the error in judged depth, does not 
resolve its cause. A change in the apparent nearness of 
the virtual object due to its appearing to occlude the 
nearby checkerboard could be the cause of the measured 
convergence rather than its consequence. This question 
can only be resolved experimentally. 

5. Experiment 4 

One approach to analyzing whether the oculomotor ef- 
fect, i.e. the convergence, observed in Experiment 3 is 
caused by the superposition of the virtual object on the 
background is to devise a stimulus condition which on 
one hand strengthens the oculomotor cues to convergence 
but on the other hand weakens the visual evidence for oc- 
clusion, thus reducing the likelihood of convergence 
caused by proximal vergence. 

We have attempted to create such a stimulus by cutting 
an annular slot 8 cm wide out of our rotating checker- 
board so that the virtual pyramid would be just able to 
"fall through" the resulting hole (see Figure 6) . The 



outer rim of the checkerboard was supported by thin radial 
wire matching the color of the background wall and 
therefore being invisible to the observers. This stimulus 
triples the number of moving edges that provide the 
strong disparity discontinuity which could be the stimu- 
lus to convergence that could be the cause of the change 
in static convergence observed in Experiment 3. If the 
better stimulus to convergence provided by the checker- 
board were the cause of the change in static convergence, 
this stimulus should strengthen the effect On the other 
hand, the slotted hole virtually eliminates the visual evi- 
dence for occlusion. If proximal vergence triggered by 
occlusion were the cause of the change in judged distance, 
one would expect not to find a change in the judged dis- 
tance of the virtual object when it is presented in the slot 



Figure 6. Slotted physical surface 

5.1 Methods 

Nine subjects aged 23 - 47 who were either laboratory 
personnel or paid subjects provided by Bionetics were 
used in Experiment 4. 

This experiment was conducted using a methodology 
equivalent to that of Experiment 2 for stereoscopic virtual 
objects. Two different depicted distances of the virtual 
objects, 83 cm and 108 cm, were randomly ordered into 
blocks of 20 runs. In each block either a solid checker- 
board or a slotted checkerboard was introduced along the 
line of sight of the virtual object. After introduction of 
the checkerboard, the change in the judged distance to the 
virtual object was measured. Block types were alternated 
for all subjects. All but one subject, from whom half of 
his data was lost, experienced 4 blocks, making a total of 
80 judgments per subject. The order of presentation of 
the two checkerboard types was counterbalanced across 
subjects. 



Figure 7. Hole effect on judged distance to virtual objects. 


5.2 Results 

Analysis of variance showed that while the solid disk 
caused a previously observed offset of the judged virtual 
object distance towards the observer (Mean: 2.80 cm; SE: 
±0.75 cm), the slotted disk caused only a mean 0.72 cm 
(SE ±0.49 cm) change. This difference was statistically 
significant. F(l,8) = 19.605 p < 0.002. The offset for 
the slotted disk condition, though quite close to 0, is sta- 
tistically significantly less than 0. (t=-2.59, df=8, p< 04) 

The size of the offset was larger for the greater depicted 
distance. The offset for the 83 cm virtual object was 1.49 
cm (SE ±0.43) and 2.03 cm (SE ±0.57) for the 108 cm 
object. This difference was just significant (F(l,8) = 
5.349 p < 0.05). There was no statistical interaction 
(F(l,8) = 3.068 p > 0.05) so the effects of placing the 
virtual object in the slot was statistically indistinguish- 
able for the two presentation distances. Accordingly, the 
data from the two presentation distances may be collapsed 
as in Figure 7. This figure shows the full distribution of 
all the subjects responses illustrating the effect of the slot 
on the judged distance to the virtual object. 

5.3 Discussion 

As is clear from Figure 7, introduction of the slot in 
the checkerboard that removed the occlusion between the 
vir tual object and the checkerboard greatly reduced the off- 
set in judged virtual object position produced by the 
checkerboard introduction. This reduction makes the ocu- 
lomotor explanation of the change less likely since the 
binocular depth cues of the checkerboard would be ex- 
pected, if anything, to strengthen any vergence response. 
Since the overly of the virtual object contours and those 
of the checkerboard seems to be die key feature causing 
the shift in its judged position, the occlusion cue placing 
the virtual object in front of the checkerboard seems to be 
the best explanation for the nearer localization of the vir- 
tual objects. The change in static convergence, thus, ap- 
pears to be a consequence of proximal vergence and re- 
sembles effects in recent reports of convergence being 
driven by the kinetic depth effects which produce a per- 
ception of nearness [15]. 

There is, however, another possible interpretation of 
the results which could be based on the proposal that the 
alternative explanations of the offset of the judged dis- 
tance of the virtual object are not mutually exclusive 
[16]. In this view the introduction of the slotted checker- 
board stimulus could introduce a proximal vergence ten- 
dency to fixate farther away since any objects seen 
through the slot would necessarily be more distant than 
the slot. Were this tendency for a more distant fixation to 
occur, it would oppose any tendency of the visual infor- 
mation to reduce oculomotor bias. The net effect of the 
opposition could account for the almost negligible for- 
ward displacement of the virtual object observed while 
subjects were exposed to the slotted checkerboard. 




One way to dissociate the two influences could be to 
study the individual oculomotor biases of each subject, 
measuring for example their phoria with a Maddox Rod 
test or equivalent. To the extent subjects exhibit ex- 
ophoria, one could expect a exofixation disparity while 
viewing the virtual object. Its removal by presentation of 
the checkerboards could be associated with the change in 
judged distance. An explanation of the observed effects of 
the checkerboards solely in terms of proximal vergence 
would not predict a correlation between the phoria tests 
and the subjects’ individual phoria variations. 
Accordingly, future investigations could examine individ- 
ual subject’s oculomotor biases as a technique to more 
precisely determine the cause of the offset of the change 
in judged distance associated with superimposition of vir- 
tual objects against physical surfaces. 

6. Design Considerations 

1) The present results were observed with a static eye 
point and can be expected to change when significant lat- 
eral head-movement producing motion parallax is intro- 
duced. Nevertheless, it is important to realize that since 
many of the new applications of head-mounted see- 
through displays, in fact, will involve relatively static 
viewing, the conditions used remain practically relevant. 

2) Since weight and cost considerations may argue for the 
use of monocular displays, they are likely to be initial 
candidates for many applications. Accordingly, such dis- 
plays should have a variable focus control to appropri- 
ately direct the convergence of prepresbyopic users. 
Designers and supervisors should be aware that operators 
over 40 will generally not benefit from the variable focus 
adjustment. 

3) Biocular and stereo displays should be used with a 
bore-sighting procedure in which focus is adjusted to a 
reference target so as to correct for any errors in depth due 
to inappropriate vergence. 

4) Computer generated or other targets presented binocu- 
larly should have individually tailored stereo disparity to 
correct their spatial localization so as to compensate for 
the tendency of virtual objects to appear to float in front 
of the surfaces that they are seen against. 
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